55 research outputs found
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Learning based methods have shown very promising results for the task of
depth estimation in single images. However, most existing approaches treat
depth prediction as a supervised regression problem and as a result, require
vast quantities of corresponding ground truth depth data for training. Just
recording quality depth data in a range of environments is a challenging
problem. In this paper, we innovate beyond existing approaches, replacing the
use of explicit depth data during training with easier-to-obtain binocular
stereo footage.
We propose a novel training objective that enables our convolutional neural
network to learn to perform single image depth estimation, despite the absence
of ground truth depth data. Exploiting epipolar geometry constraints, we
generate disparity images by training our network with an image reconstruction
loss. We show that solving for image reconstruction alone results in poor
quality depth images. To overcome this problem, we propose a novel training
loss that enforces consistency between the disparities produced relative to
both the left and right images, leading to improved performance and robustness
compared to existing approaches. Our method produces state of the art results
for monocular depth estimation on the KITTI driving dataset, even outperforming
supervised methods that have been trained with ground truth depth.Comment: CVPR 2017 ora
Indirect 3D Reconstruction Through Appearance Prediction
As humans, we easily perceive shape and depth, which helps us navigate our environment and interact with objects around us. Automating these abilities for computers is critical for many applications such as self-driving cars, augmented reality or architectural surveying. While active 3D reconstruction methods, such as laser scanning or structured light can produce very accurate results, they are typically expensive and their use cases can be limited. In contrast, passive methods that make use of only easily captured photographs, are typically less accurate as mapping from 2D images to 3D is an under-constrained problem. In this thesis we will focus on passive reconstruction techniques. We explore ways to get 3D shape from images in two challenging situations: 1) where a collection of images features a highly specular surface whose appearance changes drastically between the images, and 2) where only one input image is available. For both cases, we pose the reconstruction task as an indirect problem. In the first situation, the rapid change in appearance of highly specular objects makes it infeasible to directly establish correspondences between images. Instead, we develop an indirect approach using a panoramic image of the environment to simulate reflections, and recover the surface which best predicts the appearance of the object. In the second situation, the ambiguity inherent in single-view reconstruction is typically solved with machine learning, but acquiring depth data for training is both difficult and expensive. We present an indirect approach, where we train a neural network to regress depth by performing the proxy task of predicting the appearance of the image when the viewpoint changes. We prove that highly specular objects can be accurately reconstructed in uncontrolled environments, producing results that are 30% more accurate compared to the initialisation surface. For single frame depth estimation, our approach improves object boundaries in the reconstructions and significantly outperforms all previously published methods. In both situations, the proposed methods shrink the accuracy gap between camera-based reconstruction versus what is achievable through active sensors
Digging Into Self-Supervised Monocular Depth Estimation
Per-pixel ground-truth depth data is challenging to acquire at scale. To
overcome this limitation, self-supervised learning has emerged as a promising
alternative for training models to perform monocular depth estimation. In this
paper, we propose a set of improvements, which together result in both
quantitatively and qualitatively improved depth maps compared to competing
self-supervised methods.
Research on self-supervised monocular training usually explores increasingly
complex architectures, loss functions, and image formation models, all of which
have recently helped to close the gap with fully-supervised methods. We show
that a surprisingly simple model, and associated design choices, lead to
superior predictions. In particular, we propose (i) a minimum reprojection
loss, designed to robustly handle occlusions, (ii) a full-resolution
multi-scale sampling method that reduces visual artifacts, and (iii) an
auto-masking loss to ignore training pixels that violate camera motion
assumptions. We demonstrate the effectiveness of each component in isolation,
and show high quality, state-of-the-art results on the KITTI benchmark.Comment: ICCV 1
Heightfields for Efficient Scene Reconstruction for AR
3D scene reconstruction from a sequence of posed RGB images is a cornerstone task for computer vision and augmented reality (AR). While depth-based fusion is the foundation of most real-time approaches for 3D reconstruction, recent learning based methods that operate directly on RGB images can achieve higher quality reconstructions, but at the cost of increased runtime and memory requirements, making them unsuitable for AR applications. We propose an efficient learning-based method that refines the 3D reconstruction obtained by a traditional fusion approach. By leveraging a top-down heightfield representation, our method remains real-time while approaching the quality of other learning-based methods. Despite being a simplification, our heightfield is perfectly appropriate for robotic path planning or augmented reality character placement. We outline several innovations that push the performance beyond existing top-down prediction baselines, and we present an evaluation framework on the challenging ScanNetV2 dataset, targeting AR tasks
Etude de l'influence de la viscosité sur la distribution de taille de gouttes par mesure PDPA
International audienceLe diagnostic de Phase Doppler Anemometry est une technique de mesure largement utilisĂ©e pour Ă©tudier les distributions en taille et en vitesse de gouttes dans un spray, mais celui-ci est rarement appliquĂ© sur des fluides visqueux car trĂšs sensible Ă la non sphĂ©ricitĂ© des gouttes formĂ©es. La prĂ©sente Ă©tude se propose dâĂ©tudier les performances de ce diagnostic sur un injecteur industriel de type simplex fonctionnant avec deux carburants, du kĂ©rosĂšne et un mĂ©lange huile/kĂ©rosĂšne permettant de simuler la viscositĂ© du kĂ©rosĂšne en haute altitude.CouplĂ©e Ă ces mesures, la technique dâombroscopie ultra rapide a Ă©tĂ© utilisĂ©e pour analyser la structure du spray, donner une indication sur les conditions Ă Ă©tudier et post-traiter les mesures de PDA donc le procĂ©dĂ© de validation des mesures nâĂ©tait pas satisfaisant
Constant Velocity Constraints for Self-Supervised Monocular Depth Estimation
We present a new method for self-supervised monocular depth estimation. Contemporary monocular depth estimation methods use a triplet of consecutive video frames to estimate the central depth image. We make the assumption that the ego-centric view progresses linearly in the scene, based on the kinematic and physical properties of the camera. During the training phase, we can exploit this assumption to create a depth estimation for each image in the triplet. We then apply a new geometry constraint that supports novel synthetic views, thus providing a strong supervisory signal. Our contribution is simple to implement, requires no additional trainable parameter, and produces competitive results when compared with other state-of-the-art methods on the popular KITTI corpus
A far-ultraviolet-driven photoevaporation flow observed in a protoplanetary disk
Most low-mass stars form in stellar clusters that also contain massive stars,
which are sources of far-ultraviolet (FUV) radiation. Theoretical models
predict that this FUV radiation produces photo-dissociation regions (PDRs) on
the surfaces of protoplanetary disks around low-mass stars, impacting planet
formation within the disks. We report JWST and Atacama Large Millimetere Array
observations of a FUV-irradiated protoplanetary disk in the Orion Nebula.
Emission lines are detected from the PDR; modelling their kinematics and
excitation allows us to constrain the physical conditions within the gas. We
quantify the mass-loss rate induced by the FUV irradiation, finding it is
sufficient to remove gas from the disk in less than a million years. This is
rapid enough to affect giant planet formation in the disk
SimpleRecon: 3D reconstruction without 3D convolutions
Traditionally, 3D indoor scene reconstruction from posed images happens in two phases: per-image depth estimation, followed by depth merging and surface reconstruction. Recently, a family of methods have emerged that perform reconstruction directly in final 3D volumetric feature space. While these methods have shown impressive reconstruction results, they rely on expensive 3D convolutional layers, limiting their application in resource-constrained environments. In this work, we instead go back to the traditional route, and show how focusing on high quality multi-view depth prediction leads to highly accurate 3D reconstructions using simple off-the-shelf depth fusion. We propose a simple state-of-the-art multi-view depth estimator with two main contributions: 1) a carefully-designed 2D CNN which utilizes strong image priors alongside a plane-sweep feature volume and geometric losses, combined with 2) the integration of keyframe and geometric metadata into the cost volume which allows informed depth plane scoring. Our method achieves a significant lead over the current state-of-the-art for depth estimation and close or better for 3D reconstruction on ScanNet and 7-Scenes, yet still allows for online real-time low-memory reconstruction. Code, models and results are available at https://nianticlabs.github.io/simplerecon
Influence de la variabilitĂ© des dĂ©bits sur les taux dâĂ©rosions etle relief long-terme : lâexemple de la bordure sud-est du MassifCentral, France
National audienceLâĂ©volution des paysages rĂ©sulte de lâaction conjointe des forçages tectoniques etclimatiques. Ces processus nâagissent pas de maniĂšres continus mais via desĂ©vĂ©nements ponctuels (sĂ©ismes, glissements de terrains, crues majeurs) qui, intĂ©grĂ©ssur des temps longs conduisent Ă la formation des reliefs. Lâincision fluviale contrĂŽle ladĂ©nudation des paysages et, est souvent modĂ©lisĂ©e comme une fonction de la contraintede cisaillement et de la puissance du cours dâeau. Ce type de modĂšle (Stream PowerModel) exprime lâincision des riviĂšres en fonction de lâaire de drainage et de la pente duchenal qui sont des variables facilement quantifiables Ă partir des donnĂ©estopographiques. NĂ©anmoins, il ne prend pas en compte certains paramĂštres tels que leseuil dâincision et la variabilitĂ© des dĂ©bits, ce qui a nĂ©cessitĂ© des Ă©volutions de cemodĂšle (Stochastic âŠ) devant encore ĂȘtre validĂ©es par des donnĂ©es de terrain.La bordure sud-est du Massif Central est une zone intĂ©ressante pour Ă©tudier cesproblĂ©matiques car elle prĂ©sente des Ă©pisodes de fortes prĂ©cipitations concentrĂ©es surle relief, entraĂźnant des diffĂ©rences marquĂ©es dans les distributions des dĂ©bits. Noustestons ces modĂšles en quantifiant les taux dâĂ©rosion Ă lâaide des nuclĂ©idescosmogĂ©niques (10Be), en caractĂ©risant la variabilitĂ© des dĂ©bits avec les stationshydromĂ©triques et en effectuant une analyse morphologique des profils de riviĂšres.L'analyse de 326 stations hydromĂ©triques nous permet d'observer un fort gradient devariabilitĂ© des dĂ©bits depuis la bordure SE jusqu'Ă l'intĂ©rieur du massif. Lesconcentrations en 10Be mesurĂ©es dans les sĂ©diments des riviĂšres de 34 bassins versantsimpliquent une grande variation des taux dâĂ©rosion entre 29 et 126 mm/ka. Nouscomparons ces taux avec diverses paramĂštres hydro-morphologiques et, intĂ©grons cesobservations dans le cadre des modĂšles. Nos rĂ©sultats confirment lâexistence desrelations non linĂ©aires entre les taux de dĂ©nudation et le steepness index et leurdĂ©pendance Ă la variabilitĂ© hydrologique et au runoff
Influence de la variabilitĂ© des dĂ©bits sur les taux dâĂ©rosions etle relief long-terme : lâexemple de la bordure sud-est du MassifCentral, France
National audienceLâĂ©volution des paysages rĂ©sulte de lâaction conjointe des forçages tectoniques etclimatiques. Ces processus nâagissent pas de maniĂšres continus mais via desĂ©vĂ©nements ponctuels (sĂ©ismes, glissements de terrains, crues majeurs) qui, intĂ©grĂ©ssur des temps longs conduisent Ă la formation des reliefs. Lâincision fluviale contrĂŽle ladĂ©nudation des paysages et, est souvent modĂ©lisĂ©e comme une fonction de la contraintede cisaillement et de la puissance du cours dâeau. Ce type de modĂšle (Stream PowerModel) exprime lâincision des riviĂšres en fonction de lâaire de drainage et de la pente duchenal qui sont des variables facilement quantifiables Ă partir des donnĂ©estopographiques. NĂ©anmoins, il ne prend pas en compte certains paramĂštres tels que leseuil dâincision et la variabilitĂ© des dĂ©bits, ce qui a nĂ©cessitĂ© des Ă©volutions de cemodĂšle (Stochastic âŠ) devant encore ĂȘtre validĂ©es par des donnĂ©es de terrain.La bordure sud-est du Massif Central est une zone intĂ©ressante pour Ă©tudier cesproblĂ©matiques car elle prĂ©sente des Ă©pisodes de fortes prĂ©cipitations concentrĂ©es surle relief, entraĂźnant des diffĂ©rences marquĂ©es dans les distributions des dĂ©bits. Noustestons ces modĂšles en quantifiant les taux dâĂ©rosion Ă lâaide des nuclĂ©idescosmogĂ©niques (10Be), en caractĂ©risant la variabilitĂ© des dĂ©bits avec les stationshydromĂ©triques et en effectuant une analyse morphologique des profils de riviĂšres.L'analyse de 326 stations hydromĂ©triques nous permet d'observer un fort gradient devariabilitĂ© des dĂ©bits depuis la bordure SE jusqu'Ă l'intĂ©rieur du massif. Lesconcentrations en 10Be mesurĂ©es dans les sĂ©diments des riviĂšres de 34 bassins versantsimpliquent une grande variation des taux dâĂ©rosion entre 29 et 126 mm/ka. Nouscomparons ces taux avec diverses paramĂštres hydro-morphologiques et, intĂ©grons cesobservations dans le cadre des modĂšles. Nos rĂ©sultats confirment lâexistence desrelations non linĂ©aires entre les taux de dĂ©nudation et le steepness index et leurdĂ©pendance Ă la variabilitĂ© hydrologique et au runoff
- âŠ